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[57] ABSTRACT 

A head-mounted image display device with a good oper- 
abilily and a data processing apparatus including such a 
display device. A head-mounted image display device 
wherein a liquid crystal panel and an enlarging optical 
system are housed in a frame and wherein images generated 
on the liquid crystal display panel through the enlarging 
optical system are visible in the field of view in front of the 
frame comprises a first direction detection means (107) 
disposed in part of the frame (101) for detecting the direction 
of the user's head; an observation point detection means 
(109) disposed in part of the frame for detecting the user's 
line of sight; a trigger input means (4) for transmitting 
trigger signals; and an image generation means (3, 105) for 
controUing images generated on the liquid crystal display 
panel based on signals from the first direction detection, 
observation point detection, and trigger input means. 

6 Claims, 39 Drawing Sheets 
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HEAD-MOUOTED IMAGE DISPLAY DEVICE 
AND DATA PROCESSING APPARATUS 
INCLUDING THE SAME 

BACKGROUND OF THE INVENTION 5 

1. Field of the Invention 

The present invention relates to a head-mounted image 
display device for displaying images on a screen showing a 
virtual space and to a data processing apparatus including 
this device. 

2. Description of the Prior Art 

The design of electronic circuits or experiment thereon 
requires materials such as data sheels due to the need to 
confirm the specifications of parts to be used. In this case, 15 
the materials are scattered on and around the designer's 
desk, thereby limiting the working space or hindering the 
designer from finding required materials quickly. This 
reduces the working efi&ciency. 

To carry out design efiSciently, computer aided engineer- 
ing (CAE) for working out designs \ising computers has 
been introduced. That is, a paperless design environment has 
been established in which, for example, materials such as 
data sheets are registered in a computer to enable them to be 
displayed and referenced on the screen of the display as ^ 
required. The above inconveniences have thus partly been 
eliminated. 

If, however, a large number of parts are to be used and a 
large number of materials must be referenced at the same 
time, although several windows can be opened on the screen 
to display different materials in the different windows, the 
windows must sometimes overlap each other if a large 
number of windows must be used simultaneously due to the 
physically limited size and resolution of the CRT display for 
displaying information. As a result, the amount of informa- 
tion that can be referenced at the same time is limited, and 
the efficiency provided by the introduction of CAE is thus 
hmited; in some cases, it is faster to find required materials 
among those scattered on and around the table as described 
above. 

The working efficiency is also reduced in the maintenance 
or repair of an installed apparatus, wherein the operator must 
carry out such an operation while referencing instruction 
manuals and wherein because the surroundings are in 
disorder, the operator cannot place the manuals in positions 
in which they can be referenced appropriately. Under such 
conditions, however, it is further difficult to place a CRT 
display close to the operator, and the introduction of CAE is 
impossible. Thus, no other appropriate solutions were avail- 
able. 

Several methods of mounting a display apparatus on the 
operator's head and displaying information from materials 
thereon have thus been proposed. 

Published Unexamined Patent Application No. 5-100192 ss 
describes a method of displaying screens from electronics 
such as word processors on a spectacle -like display, but this 
is only a spectacle-shaped variation of conventional displays 
and is not suited to the presentation of a large number of 
materials. E^ibhshed Unexamined Patent Application No. go 
4-263283 proposes a portable virtual reahty apparatus, but 
this simply provides an artificial reality apparatus and is not 
suited to the manipulation of displayed data. 

In addition, for conventional head-mounted image display 
devices, several problems arising from the need to altema- 65 
tively view a virtual image on the liquid crystal display and 
the corresponding image on the external keyboard due to the 
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large difference in image quality between these images, have 
been pointed out. To solve this problem, some conventional 
techniques cause a display image and the corresponding 
external image to overlap each other. 

Head-mounted image display devices of this type, 
however, also have problems. For example, in such 
apparatuses, the visibility is very bad because external 
images are simply caused to overlap the corresponding 
display images, and alternatively viewing different images 
may strain the operator's eyes. 

In addition, in head-mounted image display devices, 
when, for example, data is input to a word processor, keys 
to be pressed must be confirmed while viewing a display 
image, so the external keyboard must be checked while 
watching the display image. 

To solve this problem, a means for switching between an 
external image and the corresponding image from an arith- 
metic operational device may be provided, or a configura- 
tion that enables the use of part of the fi.eld of view for 
watching the display while using the other part for checking 
the exterior may be provided, as proposed in PubUshed 
Unexamined Patent Application No. 5-100192. 

In this case, however, since data must be input while using 
a particular part of the field of view to check the input 
device, mounting the image display device on the operator's 
bead eliminates the freedom of the operator's inputting 
position, and long time work thus results in a significant 
fatigue. In addition, if a keyboard is used as an input device, 
the input operation is very difficult when the keys are larger 
than the tips of the operator's fingers, so the limit of the 
practical size of portable terminals depends on the size of the 
keyboard. Furthermore, due to the fixed arrangement of keys 
in conventional keyboards, optimal key arrangements for 
various applications are not available. In this information- 
oriented society, more and more attention is being paid to 
multimedia applications, so people must often deal with data 
in which motion and still pictures are mixed. Under these 
circumstances, the need of complicated procedures for cre- 
ating motion pictures hinders people without expertise from 
providing adequate expressions using multiple media effec- 
tively. 

For hypermedia applications, HyperCard commercially 
available from Apple Co., Ltd is popular, but requires a large 
number of mouse clicks, thereby possibly resulting in mis- 
takes in clicking positions. 

For movie postproduction processes for processing, 
synthesizing, and editing movie materials, a large amount of 
time is required to complete the process due to the need to 
accurately carry out complex procedures according to 
precalculations, and the flexibility of these processes is 
limited; it is impossible to test various combinations of 
images during the process. 

In addition, if a head-mounted image display device is 
used, only the person wearing the device can view image 
display information output from a data processing apparatus. 
If a plurality of people have to view image display infor- 
mation output from the data processing apparatus, then: 

(1) these people must use a single head-mounted image 
display device in turn, or 

(2) another data processing apparatus with a CRT display 
must be provided. 

In this manner, a plurality of devices may be required 
according to a particular purpose and usage. 

Attempts are being made to reduce the size of computers 
and word processors, and the utilization of portable data 
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processing apparatuses is increasing. Such portable data 
processing apparatuses ordinarily include an open type 
display that is opened in use. However, with the further 
reduction of the size of portable data processing apparatuses, 
the extension of the range within which such devices can be 
used, and the increase in demand for functions of publicly 
creating confidential documents, the use of head-mounted 
image display devices has been proposed. As in Published 
Unexamined Patent Application No. 5-100192, such appa- 
ratuses comprise an arithmetic processing device and a 
head-mounted image display device to display screen infor- 
mation output from the arithmetic operation device, through 
the head-mounted image display device as enlarged virtual 
images. In such conventional structures, the arithmetic 
operation device and the head-mounted image display 
device are always separated from each other, they must be 
carried separately, resulting in poor portability despite their 
reduced sizes. Even if the head-mounted image display 
device is carried in an outer sheath case, handling becomes 
cumbersome, and faults are likely to occur. 

BRIEF SUMMARY OF THE INVENTION 

It is an objective of this invention to provide a head- 
mounted image display device suitable to efficient design, 
experiment, maintenance, and repairs. 

It is another objective of this invention to provide a 
head-mounted image display device that has good portabil- 
ity and visibility and that does not affect the user's health. 

It is yet another objective of this invention to provide a 
head-mounted image display device that enables the opera- 
tor to input data in a natural position without depending on 
the size of an input device. 

It is yet another objective of this invention to provide a 
head-mounted image display device that can be used for 
multimedia applications and that enables the user to check 
both an image display section and displayed information on 
the attributes of a displayed image simply by changing the 
direction of the field of view when editing still and motion 
pictures. 

It is still another objective of this invention to provide a 
head-mounted image display device that uses as an image 
retrieval method that is complicated when used with hyper- 
media applications a method for displaying retrieved 
attributes based on time series or hierarchies and that 
reduces mistakes attributable to the image retrieval method 
simply by switching the direction of the field of view 
between a first display area and a second display area. 

It is stiU another objective of this invention to provide a 
head-mounted image display device that enables the three- 
dimensional display of information displayed in the second 
display area the amount of which varies with changes in 
space or time by changing the parallax of the operator's 
eyes, thereby reducing mistakes in inputting hierarchical 
data. 

It is StiU another objective of this invention to provide a 
head-mounted image display device that can express even 
minor positional relationships in an image during edition in 
imaginative movie production methods in which voices are 
supplement arily input to process and edit movie materials 
and that in the edition of motion pictures, can express 
conditions based on time scries such as partial acceleration 
and deceleration in such a way that they look like real 
conditions. 

It is still another objective of this invention to provide a 
data processing apparatus that can simply switch a display 
means between a head-mounted image display device and a 
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screen or the Uke as required to display thereon image 
display information output from the data processing appa- 
ratus main body. 
It is still another objective of this invention to provide a 

5 data processing apparatus comprising a he ad -mounted 
image display device that is portable and easy to handle. 

(A) A head-mounted image display device according to 
one aspect of this invention wherein a liquid crystal panel 
and an enlarging optical system are housed in a frame and 
wherein images generated on the liquid crystal display panel 
through the enlarging optical system are visible in the field 
of view in front of the frame, comprises a first direction 
detection means disposed in part of the frame for detecting 
the direction of the user's head, an observation point detec- 
tion means disposed in part of the frame for detecting the 
user's line of sight, a trigger input means for transmitting 
trigger signals, and an image generation means for controi- 
hng images generated on the liquid crystal display panel 
based on signals from the first direction detection, observa- 
tion point detection, and trigger input means. 

According to another aspect of this invention, the above 
head-mounted image display device has a second direction 
detection means mounted on the user for detecting the 

^ direction of the user's body to transmit the detection signal 
to the image generation means. 

According to yet another aspect of this invention, in the 
above head-mounted image display device, the image gen- 
eration means drives the liquid crystal display panel to 

30 display an image on a virtual screen as a virtual image, 
selects and identifies an arbitrary image from the plurality of 
displayed images based on the output of the observation 
point and direction detection means, and specifies the iden- 
tified image as those to be manipulated on the basis of the 

35 output of the trigger input means. 

According to still another aspect of this invention, in the 
above head-mounted image display device, the image gen- 
eration means moves the arbitrary selected image to the 
positions on the virtual screen which are determined on the 

40 basis of the output of the observation point and direction 
detection means and stops the movement in response to the 
output of the trigger input means. 

With the above configuration, in designing circuits or 
experimenting thereon, this invention can display on a 

45 virtual screen fixed to the space those materials such as data 
sheets which are used to confirm the specifications of parts 
to be used, thereby preventing the working space from being 
limited by a large number of materials to avoid the reduction 
of efficiency even when a large number of parts must be 

50 used. 

In carrying out the maintenance or repair of an installed 
apparatus, since the locational relationship with the operator 
can be displayed on a fixed virtual screen, eflScient opera- 
tions are ensured even if there is no appropriate place in 
which manuals can be put. 

Furthermore, due to the ability of this invention to freely 
move arbitrary virtual screens, required screens can be 
moved to positions in which images can be seen most 
clearly, thereby eliminating the need for the operator to 
assume an unnatural posture. 

The movement of virtual screens does not require the use 
of the operator's hands, so they can be solely used for 
operations. 

65 (B) A head-mounted image display device according to 
another aspect of this invention wherein a liquid crystal 
panel and an enlarging optical system are housed in a frame 
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and wherein images generated on the liquid crystal display According to still another aspect of this invention, in the 

panel through the enlarging optical system are visible in the above head-mounted image display device, the voice input 

field of view in front of the frame, has a screen disposed in means is composed of a small microphone that enables 

the direction in which enlarged virtual images generated by unspecified speakers to input voices thereto, 

the enlarging optical system are formed. 5 addition, a head-mounted image display device accord- 

Accordmg to another aspect of this invention, the above i^g to still another aspect of this invention wherein a Hquid 

head-mounted image display device further has a synthesiz- ^al panel and an enlarging optical system are housed in 

ing optical means for synthesizmg an miage showing the ^ ^^^^ ^^^^^^ ^ g,n,mcd on the liquid crystal 

. ' t- . . r t ' ■ • . visible in the field of view in front of the frame, has a 

Accordmg to yet another aspect of this invention, the 1 j . „ r j . *u *i u ^ • 

u uj ^j- j-ij- t \ r location detection means for detecting the field of view in 

above head-mounted image display device employs part of *u u • » 1 -«* ij- ** 5 • 

. V ,v i: ui r J r horizontal or vertical directions and an image generation 

an outer sheath case for a portable computer as a screen. - . . 

TiTiU *u u .u- • • 1 . means for segmenting the display area into a first display 

With the above constitution, this mventioQ implements an rj f - • j j^-i r 

image display device that has good portability a^d visibility displaying images and a second display area for 

and that does not aSect the operator's health by providing an displaying attributes of an image and switching between the 

,1 ' „^ ^1 tu^ i T.- vi first and second display areas based on the output of the 

optical system tor syntnesizing an image on the liqmd 1 • . . • ^ 

i J . f.. ji location detection means, 
crystal display and an image of the extenor and also pro- 
viding a screen in the direction in which images on the Uquid Accordmg to stHl another aspect of this invention, in the 
crystal display are formed, or by using a single eye-type ^'^^ve head-mounted image display device, the image gen- 
head-mounted image liquid crystal display device with a ^''^^^^n means can switch between the first and second 
screen disposed in the direcUon in which vertual images on display areas based on the output of the location detection 
the liquid crystal display are formed. A portable computer controls attribute data shown in the second 
that has good poitabHity and visibility and that does not ^^^play area accordmg to voice input signals, 
affect the operator's health can also be implemented by In addition, a head-mounted image display device accord- 
employing part of an outer sheath cover of the computer as ing to still another aspect of this invention wherein a Hquid 
a screen. crystal display panel and an enlarging optical system are 

(C) A head-mounted image display device according to ^o^^ed in a frame and wherein images generated on the 
yet another aspect of this invention wherein a liquid crystal l^l^id crystal display panel through the enlarging optical 
panel and an enlarging optical system are housed in a frame 30 system are visible in the field of view in front of the frame, 
and wherein images generated on the liquid crystal display ^ location detection means for detecting the field of view 
panel through the enlarging optical system are visible in the horizontal or vertical directions and an image gen- 
field of view in front of the frame, has an input device with eration means for segmenting the display area into a first 
a sensor for inputting signals and an image generation means display area for displaying images and a second display area 
for assigning input keys to locational information firom the 35 for displaying attributes of an image and switching between 
input device and synthesizing on/off information on the fi^st and second display areas based on the output of the 
input keys and the results of arithmetic operations to allow location detection means to display in the second display 
the Hquid crystal display panel to generate synthesized ^rea for displaying attributes of an image a geometry or 
(m^gs operation model to be incorporated in the first display area, 

Accordirig to yet another aspect of this invention, the 40 According to still another aspect of this invention, in the 

above head-mounted image display device displays input above head-mounted image display device, geometrical data 

information on part of the display screen and the results of or camera work to be incorporated in the first display area is 

arithmetic operations on the other part. displayed in the second display area. 

With the above configuration, this invention enables the According to still another aspect of this invention, in the 
user to input data in a nahiral position without viewing the 45 above head-mounted image display device, space- time area 

input device, and to check the conditions of the input device data to be incorporated in the first display area is displayed 

without substantially moving the user's field of view. In in the second display area. 

addition, since a key assignment means in the arithmetic According to still another aspect of this invention, in the 

operation device executes key assignment, an optimal key- above head-mounted image display device, space-time area 
board for a particular appHcation can be provided regardless 50 data to be incorporated in the first display area is three- 

of the number of keys in the input device, and various dimensionally displayed in the second display area with a 

interface screens can be displayed in a virtual keyboard parallax provided therein. 

display section. Furthermore, the input device does not In addition, a data processing apparatus according to still 

require the size and arrangement of input keys to be deter- another aspect of this invention comprises a bead-mounted 
mined in advance, so the use of a small tablet enables a small 55 image display device wherein a Hquid crystal display panel 

portabble terminal to be realized. and an enlarging optical system are housed in a frame and 

(D) A head-mounted image display device according to wherein images generated on the liquid crystal display panel 
still another aspect of this invention wherein a Hquid crystal through the enlarging optical system are visible in the field 
panel and an enlarging optical system are housed in a frame of view in front of the frame; and a main body including a 
and wherein images generated on the liquid crystal display 60 central processing unit and a keyboard, with the image 
panel through the enlarging optical system are visible in the display device and the main body connected together via a 
field of view in front of the frame, has a location detection connector. 

means for detecting the field of view in the horizontal or According to still another aspect of this invention, in the 

vertical directions, a voice input means joined with the above data processing apparatus, the main body has a socket 
frame using a flexible joint, and an image generation means 65 in which the liquid crystal display panel is mounted, the 

for controlling images genera ted on the liquid crystal display Hquid crystal display panel mounted in the socket and the 

panel based on the location detection and voice input means. Hquid crystal display panel provided in the head-mounted 



06/13/2002, EAST Version: 1.03.0002 




5,977,935 

7 8 

image display device are each detachably constituted, and in which the liquid crystal display panel is mounted, the 

the Uquid crystal display panels arc also configured so as to Uquid crystal display panel mounted in the socket and the 

be shared by the main body and the head-mounted image Hquid crystal display panel provided in the head-mounted 

display device. image display device are each detachably constituted, and 
With the above constitution, this invention enables the 5 the Uquid crystal display panels are also configured so as to 

operator to precisely adjust the sizes of images and relative be shared by the main body and the head-mounted image 

relationships among them during edition while viewing the display device. 

images, by providing the location detection means for With the above configuration, this invention enables the 

detecting the field of view in the horizontal or vertical display means to be switched simply and quickly as 

directions and the voice input means joined with the frame required. Images displayed on the head-mounted image 

using a flexible joint. display or screen or the like are larger than those on 

Since the voice input means comprises a small micro- conventional displays, so the screen in this invention has a 

phone that enables unspecified speakers to input voices large capacity. Furthermore, since a single device can 

thereto, any user can wear this means. Furthermore, by accommodate a plurality of usages, it is not necessary to use 

inputting a form of voice input processing using WordSpot- different devices for different purposes, 

ting on a neuro-computer, mistakes in input procedures are Conventional apparatuses require two cords for supplying 

somewhat compensated for because associated keywords ^^^^^ the image display device and for transmitting 

can be recognized on the basis of certain probabilisUc ^ information to the same device because the 

determmaUon, thereby enabling edition to be executed with- ^^^.^ • ^ j ^^^-^^ ^^^^ ^^^^ ^ 

out the need to memorize all the commands. ^ ,^ ^j^^ invention, however, the connector includes 

Since the location detection means can switch between ^he power supply for driving the image display device and 

the first and second display areas, the display area can be information transmission means for transmitting image 

spatiallyclassmedintoaplurahtyofpages,therebyenabling information to the image display device, as 

the overall picture to be confirmed dunng edition, m contrast described above, so a single cord can be used to supply 

to conventional techniques wherein an image and its ^^^^ ^^^^^^ ^^^-^^ ^ ^ t^^^^^it 
aUnbutes are displayed in a single page constitulmg a ^ ^^^^^ ^^^^^^ information to the same device. 

p ay area. , , ^ , . r (F) In a data processing apparatus according to still 

Smcc the second display area for displaying attributes of ^^^^^^ ^ invention comprising a portable arith- 

an image displays a geometry or operation model to be ^^^.^ operation processing device and a head-mounted 

mcorporated in the first di^lay area, it can display not only • ^^^-^^ displaying image information from 

image materials required dunng ediUon but also materials arithmetic operation processing device, the arithmetic 

used during mteractive input, and symbols displayed m the operation device has a housing section for housing the image 

second display area can be input by clickmg on them with ^^ y^^ ^^^^^ ^^^^ ^ -^^^ ^i^pl^y ^^^^ 

a mouse, thereby reducing operational mistakes. comprises a Uquid crystal display panel provided so as to 

Since the second display area for displaying attributes of correspond to at least one of the user's eyes, an enlarging 

an image displays space-time area data to be incorporated in ^pj-^^ ^^^^^ fonning enlarged virtual images on the 

the first display area, information that varies with the lapse ^^^^^ ^^^al display, and a holding means for holding the 

of time can be processed appropriately. ^^^^^ ^.^ystal display and the enlarging optical means in 

Since the second display area for displaying attributes of fj-ont of the user's face and fixing the entire apparatus to the 
an image three-dimensionally displays hierarchical and ^ user's head. 

space-time area data to be incorporated in the first display According to still another aspect of this invention, in the 

area with a parallax provided therein, the depth of images in ^bove data processing apparatus, the arithmetic operation 

the first display area and spatial-temporal area data can be processing device has a charger having an electric contact in 

edited and processed appropriately. ^ts housing section, and the image display device has an 

Although the above display method has been described in 45 electric contact formed so as to connect to the electric 

conjunction with an embodiment that mainly displays contact in the housing section when the image display 

images and in which an image is displayed to the right of its device is housed in the housing section and a battery charged 

attributes, this invention is not limited to this arrangement vv^tien both contacts are connected together, 

and attribute data may be located to the left of the image or ^^^1^ ^bove constitution, due to its integral structure in 
over or under the image. 50 which image information from the portable arithmetic 

(E) A data processing apparatus according to still another operation processing device is displayed on the head- 
aspect of this invention comprises a head-mounted image mounted image display device, which is housed in the 
display device wherein a liquid crystal display panel and an arithmetic optation processing device, this invention enables 
enlarging optical system are housed in a firame and wherein confidential documents to be prepared, eliminates a loca- 
images generated on the liquid crystal display panel through 55 tional Umitation in which the apparatus cannot be used 
the enlarging optical system are visible in the field of view unless the display is fully opened, and improves the port- 
in front of the frame; and a main body including a central ability and oper ability of the data processing apparatus. In 
processing unit and a keyboard, with the image display addition, since the battery for driving the head-mounted 
device and the main body connected together via a connec- image display device can be charged immediately after 
tor- 60 housing, and the arithmetic operation processing device and 

According to still another aspect of this invention, in the the head-mounted image display device communicate with 

above data processing apparatus, the connector has a power each other by radio, the arithmetic operation processing 

supply for driving the image display device and an infor- device and the head-mounted image display device are 

mation transmission means for transmitting image display completely separated from each other, thereby allowing the 
information to the image display device. 65 data processing apparatus to be handled easily in its use and 

According to still another aspect of this invention, in the significantly simplifying the removal and mounting of the 

above data processing apparatus, the main body has a socket head-mounted image display device in the housing section. 



06/13/2002, EAST Version: 1.03.0002 



5,9/ 

9 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows a he ad -mounted image display device 
according to one embodiment of this invention; 

FIG. 2 is a block diagram showing the details of the 
display device in FIG. 1; 

FIG. 3 shows a scene as seen by the operator in the 
embodiment in FIG. 1; 

FIG. 4 is a block diagram showing the details of a control 
system in FIG. 2; 

FIG. 5 is a flowchart showing the operation of the control 
system in FIG. 4; 

FIG. 6 is a flowchart showing a method for specifying 
images in the embodiment in FIG. 1; 

FIG. 7 is a flowchart showing a method for moving 
images in the embodiment in FIG. 1; 

FIG. 8 shows a head-mounted image display device 
according to another embodiment of this invention; 

FIG. 9 shows a scene as seen by the operator in the 
embodiment in FIG. 8; 

FIG. 10 shows a head-mounted image display device 
according to yet another embodiment of this invention; 

FIG. 11 describes a screen applied to the embodiment in 
FIG. 10; 

FIG. 12 explains a usage of the embodiment in FIG. 10; 

FIG. 13 describes a usage of a head-mounted image 
display device according to still another embodiment of this 
invention as seen laterally; 

FIG. 14 ejfplains the usage of the embodiment in FIG. 13 
as seen frontways; 

RG. 15 describes the operator inputting data with a 
head-mounted image display device according to still 
another embodiment of this invention (which displays input 
information and the results of arithmetic operations) 
mounted on the operator's head; 

FIG. 16 explains the display screen of the image display 
device in FIG. 15; 

FIG. 17 is a block diagram showing the constitution of an 
arithmetic operation device in the embodiment in FIG. 15; 

FIG. 18 is a flowchart showing the flow of signals in the 
embodiment in FIG. 15; 

FIG. 19 is a flowchart showing a process executed in an 
assignment means in FIG. 17; 

FIG. 20 shows the configuration of the image display 
device in the embodiment in FIG. 15; 

FIG. 21 is a perspective view shovmg a data processing 
apparatus, according to one embodiment of this invention, 
which is used for a personal computer and a spectacle -like 
display used as a display means that is an example of the 
head-mounted image display device; 

FIG. 22 is a side view showing the details of a head- 
mounted image display device for the data processing appa- 
ratus in FIG. 21; 

FIG. 23 is a perspective view showing a conventional data 
processing apparatus in which a main body and a display 
section can be separated from each other; 

FIGS. 24 and 25 show the relationship between the 
vertical movement of a liquid crystal display panel and the 
position of an image; 

FIGS. 26 and 27 showy the relationship between the 
longitudinal movement of a liquid crystal display panel and 
the size of an image; 

FIG. 28 is a perspective view showing a support mecha- 
nism for the Uquid crystal display panel; 
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FIG. 29 is an exploded perspective view of FIG. 28; 

FIG. 30 is a perspective view showing a conventional data 
processing apparatus in which a main body and a display 
section can be separated from each other; 
^ FIG. 31 is a perspective view showing a data processing 
apparatus to which an image display device according to still 
another embodiment of this invention can be connected; 

FIG. 32 describes a data processing apparatus according 
to stiU another embodiment of this invention; 

FIG. 33 is a block diagram of a control system in the 
embodiment in FIG. 32; 

FIG. 34 is a flowchart describing the control system in the 
embodiment in FIG. 32; 
15 FIG. 35 shows a head-raoimted image display device 
according to still another embodiment of this invention; 

FIGS. 36 and 37 are a front and side views of an optical 
system respectively in the embodiment in FIG. 35; 

FIGS. 38 and 39 show a display method tised when the 
20 head-mounted image display device is used to edit still 
pictures. FIG. 38 is a model drawing showing the display 
method of this invention in which an image and its attributes 
are displayed in a first and second display areas that are 
located on the right and left of the screen, respectively; 
^ FIG. 40 is a flowchart showing a process executed in the 
embodiment in FIG. 35; 

FIG. 41 is a flowchart describing a voice input procedure 
in the embodiment in FIG. 35; 

FIG. 42 is a model drawing showing an embodiment of 
the display method of this invention wherein the head- 
mounted image display device according to this invention is 
used to simulate motion pictures and wherein an image and 
its attributes are displayed in a first and second display areas 
that are located on the right and left of the screen, respec- 
tively. FIG. 42(fl) shows the results of a simulated image. 
FIG, 42(6) shows the trace of the airplane. FIG. 42(c) shows 
the speed of the airplane. FIG. ^2j(d) shows the hierarchical 
segmentation of the image segments; 

FIG. 43 is a model drawing showing a data processing 
apparatus according to still another embodiment of this 
invention; 

FIG. 44 is a block diagram showing a control system in 
the data processing apparatus in FIG. 43; 
45 FIG. 45 is a model drawing describing a data processing 
apparatus according to still another embodiment of this 
invention; 

FIG. 46 is a model drawing describing a data processing 
apparatus according to still another embodiment of this 
50 invention; and 

FIG. 47 is a block diagram showing a control system in 
the data processing apparatus in FIG. 46. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

Embodiment 1 

FIG. 1 shows an embodiment in which an operator 1 is 
sitting on a chair 8 to perform operation on a desk 7. The 
operator 1 has a head-mounted image display device 2 

60 (hereafter referred to as a "display device") mounted on his 
or her head. A computer 3 is disposed under the desk 7 to 
transmit image display information to the display device 2 
via a connection cord (see FIG. 2). Furthermore^ the com- 
puter 3 has connected thereto a foot switch 4 that acts as a 

65 trigger input means. 

As shown in FIG. 2, the display device 2 has a spectacles- 
like frame 101 in which a drive circuit 105 is housed, and 
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one end of the connection cord 106 is connected to the drive 
circuit 105. The drive circuit 105 and the computer 3 
constitute an image generation means according to this 
invention. The frame 101 also has mounted in front thereof 
a liquid crystal panel 102 comprising a lateral pair of liquid 5 
crystal display devices and a back light 104 that illuminates 
the liquid crystal panel 102 rearways, and an enlarging lens 
103 is disposed on the inner surface of the frame 101 
between the liquid crystal panel 102 and the operator's eyes 
120. Image display information sent from the computer 3 via lo 
the connection cord 106 is decoded by the drive circuit 105 
and displayed on the liquid crystal panel 102. 

The liquid crystal panel 102 is of a very small full color 
or monochrome type with a screen size of about 0.7 to 1.3 
inch, and an image is enlarged by the enlarging lens 103 to 15 
about 14 inch at a visual range of 0.5 to 1.0 m and viewed 
by the operator 1 as a virtual image. The liquid crystal panel 
102, the back light 104, and the enlarging lens 103 are each 
provided for each of the operator's eyes 120, and ordinarily 
arranged so as to display the same screen to allow the eyes 20 
to see a single display screen. 

Hie display screen is located so as to be seen by the 
operator when he or she looks to a direction somewhat above 
the immediate front shown by arrow Ain FIG. 2, and a lower 
part 122 has a high transmittance within the range of about ^ 
70 to about 95% so that when looking somewhat downward 
as shown by arrow B, the operator can look near his or her 
hands through the lower part 122 of the front surface of the 
frame 101. The upper part 123 of the front surface of the 
frame 101 has a lower transmittance within the range of 0 to 
about 10% so as to prevent extraneous light from entering 
the frame. 

Furthermore, the frame 101 has mounted thereon an angle 
sensor 107 that delects the direction and angle of the 
operator's head 1 when it moves vertically or horizontally 
and that transmits detected data to the computer 3 via the 
connection cord 106. An infrared ray LED 108 is mounted 
over the enlarging lens 103, and a photosensor 109 is 
mounted under the lens 103 to detect the direction of the 

40 

operator's line of sight and winking. 

Referring FIG. 1 again, when looking somewhat upward, 
the operator 1 can view a display screen on the liquid crystal 
panel 102 as a virtual subscreen 5 on a virtual display screen 
6 at a distance about 0.5 m to 1,0 m apart from the operator, 45 
and when looking somewhat downward, he or she can see an 
operation on the desk 7. 

FIG. 3 shows a scene as seen by the operator 1. In this 
figure, both the entire virtual subscreen 5b and the lower part 
of a virtual subscreen 5c which are contained within the 50 
range shown by a display frame 51 corresponding to the 
maximum display range of the liquid crystal panel 102 are 
shown somewhat above the field of view. The desk 7 at 
which the operator 1 is performing an operation and the 
operator's hands 131 are shown somewhat below the field of 55 
view. When the operator 1 turns his or her head to the right, 
the display device 2 also moves to the right, and the display 
frame 51 accordingly moves to the right. The angle sensor 
107 detects the direction and angle of the operator's head, 
and the virtual subscreen 5 from the computer 3 scrolls 60 
leftward according to the detected angle. Tlie virtual sub- 
screen within the display frame 51 changes to the subscreen 
5c, then changes to the subscreen Sd, . . . ^ and so on. This 
is also true in the vertical direction. In this case, if the output 
of the angle sensor 107 and the distance over which the 65 
virtual subscreen 5 scrolls have been adjusted in advance, a 
fixed relative locational relationship can be maintained 
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between the desk 7 and the virtual subscreen 5. By moving 
his or her head, the operator 1 can view part of a large 
number of virtual subscreens 5; it appears to the operator as 
if the subscreens were stuck to the virtual display screen 6 
fixed to the space. 

The drive circuit 105 in the display device 2 comprises a 
CPU 132, an image storage device 133, and an image 
memory 134, as shown in FIG. 4. The image storage device 
133 has image data from the computer 3 stored therein, and 
the image memory 134 stores image data required to display 
a single screen (corresponding to the display frame 51) on 
the liquid crystal panel 102. The image storage device 133 
(and the CPU 132) may be built into the computer 3. 

The flowchart in FIG. 5 shows the operation of a control 
system in FIG. 4. The image storage device 133 regularly 
loads image data from the computer 3, and stores, for 
example, image data for the virtual subscreens 5a to 5e 
shown in FIG. 3. The CPU 132 checks whether or not the 
output of the angle sensor 107 has changed, and if it changes 
when the operator 1 moves his or her head, computes an 
address required to display a screen within the display frame 
51 according to the change in angle to output it to the image 
storage device 133. The image storage device 133 cuts 
image data corresponding to the address to output it to the 
image memory 134. The liquid crystal panel 102 displays 
the image data. In this manner, the operator 1 can view a 
large number of virtual subscreens 5 simply by moving his 
or her head. 

Next, a method for moving any one of the large number 
of virtual subscreens 5 on the virtual display screen 6 is 
described. 

First, a method for specifying any one of the large number 
of virtual subscreens 5 is described on the basis of the 
flowchart in FIG. 6. 

The position of the display frame 51 is determined by the 
output of the angle sensor 107 as described above. That is, 
this output determines the address in the image memory 134 
which corresponds to the virtual display screen 6. In 
addition, the photosensor 109 can detect what portion within 
the display frame 51 the operator is viewing. Thus, the 
photosensor enables the identification of the portion on the 
virtual display screen 6 which the operator is viewing, that 
is, the portion that corresponds to a particular address in the 
image memory 134. When the operator stamps on the foot 
switch 4 that acts as a trigger input means, the identified 
address is confirmed to specify the virtual subscreen 5 
including the address, resulting the movement of the sub- 
screen (this is like the movement of a window on the 
Windows screen using a mouse. The movement of the 
operator's line of sight corresponds to the movement of the 
mouse whereas the foot switch corresponds to the click 
button of the mouse). Thus, when stamping on the foot 
switch 4 while viewing an arbitrary virtual subscreen 5 to be 
moved, the operator can specify the portion within the 
virtual subscreen 5 which the operator 1 is viewing. 

Next, a method for moving the specified virtual subscreen 
5 is described on the basis of the flowchart in FIG. 7. 

The virtual subscreen 5 specified in the above manner 
moves on the basis of the moving direction and distance 
determined from the most recent output of the photosensor 
109 and the angle sensor 107. That is, the direction of the 
operator's line of sight is determined from the output of the 
photosensor 109 to determine where to move the virtual 
subscreen 5 within the display frame 51, and the direction of 
the operator's head is detected from the output of the angle 
sensor 107 to determine where to move the virtual subscreen 
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5 on the virtual screen 6. The virtual subscreen 5 is thus 
moved, and the destination is updated until a trigger signal 
is input. When the operator stamps on the foot switch 4, the 
specified virtual subscreen 5 stops moving and is then fixed 
to the operator's observation point. In this manner, an 5 
arbitrary virtual subscreen 5 can be moved freely to a desired 
position on the virtual screen 6. 

It is then assumed that the operator 1 is experimenting on 
an electronic circuit on the desk 7. 

Specifications and other materials for parts to be used in 
the experiment are registered in the computer 3 in advance 
to display them as a plurality of virtual subscreens 5 on the 
virtual screen 6. The operator 1 can view any material by 
displaying a desired virtual subscreen 5 within the display 
frame 51. Furthermore, stamping on the foot switch 4 while 
viewing a desired virtual subscreen 5 causes that virtual 
subscreen 5 to be specified, moving the operator's head 
enables it to be moved to a position in which it can be seen 
most clearly, and stamping on the foot switch 4 again 
enables it to be fixed on the virtual screen 6. 

If the display is complicated due to a large number of 
virtual subscreens 5 on the virtual screen 6, after specifying 
the virtual subscreen 5 as described in FIG. 6, the combi- 
nation of the number of times of stamps on the foot switch 
4 and the interval between the stamps can be adjusted so as ^ 
to send to the computer 3 an instruction for iconizing 
specified virtual subscreens 5 to reduce their sizes, thereby 
obtaining a clear display. 

If displayed virtual subscreens 5 are too small to see them 
clearly, the combination of the number of times of stamps on 
the foot switch 4 and the interval between the stamps can be 
adjusted so as to send to the computer 3 an instruction for 
enlarging specified virtual subscreens 5. 

As described above, this embodiment enables a large 
number of materials to be referenced without putting the 
surroundings in disorder, thereby avoiding the decrease in 
working efficiency. In addition, since operation can be 
achieved simply by moving the operator's head, eyes, and 
foot, and his or her hands are not occupied in the operation, ^ 
accidents such as falling of a part being experimented on or 
damage thereto can be avoided even during a complicated 
experiment. Furthermore, since required materials can be 
placed in positions in which they can be seen most clearly, 
the movement of eyes between a part being experimented on 
and the relevant material can be minimized to reduce the 
operator's fatigue. 

Even if a large number of virtual subscreens are 
displayed, they can be distributed over the virtual screen and 
are enabled to scroll by movement of the operator's head, 
thereby preventing them from overlapping each other on the 
screen. 

Furthermore, by connecting instrument used for the 
experiment to the computer 3 in advance, the results of 
measurement can be displayed close to the operator as a 55 
virtual subscreen 5 to enable the experiment to be conducted 
more efficiently. 

Embodiment 2 

FIG. 8 shows an embodiment in which the operator 1 is 
repairing an apparatus 22. The display device 2 is mounted 60 
on the operator's head, and an angle sensor 21 is mounted 
on the operator's chest, and a portable computer 13 and a 
push switch that acts as a trigger input means are mounted 
on the operator's waist. The display device 2 is the same as 
in Embodiment 1, and its description is thus omitted. The 65 
display device 2 is connected to the computer 13 through a 
connection cord. 
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In looking somewhat upward, the operator 1 can view the 
display screen of a liquid crystal panel 102 as a virtual 
subscreen 5 on a virtual display screen 6, while in looking 
somewhat downward, the operator can view the apparatus 
22 being repaired. 

FIG. 9 shows a scene as seen by the operator 1. In this 
figure, both the entire virtual subscreen Sb and the lower part 
of a virtual subscreen 5c which are contained within the 
range shown by a display frame 51 corresponding to the 
maximum display range of the liquid crystal panel 102 are 
shown somewhat above the field of view. The apparatus 
being repaired is shown somewhat below the field of view. 
When the operator 1 tums his or her head to the right, the 
display device 2 moves to the right, and the display frame 51 
accordingly moves to the right. The virtual subscreen 5 from 
the computer 13 then scrolls leftward according to the angle 
detected by an angle sensor 107 provided in the display 
device 2. The virtual subscreen within the display frame 51 
changes to the subscreen 5c, then changes to the subscreen 
5dy . . . , and so on. This is also true in the vertical direction. 
In this case, if the output of the angle sensor 107 and the 
distance over which the virtual subscreen 5 scrolls have been 
adjusted in advance, a fixed relative locational relationship 
can be maintained between the surrounding space and the 
virtual subscreen 5. It appears to the operator as if the large 
number of subscreens 5 were stuck to the virtual display 
screen 6 fixed to the space. 

If the operator 1 stoops or moves to repair another part of 
the apparatus 22, the angle sensor 21 mounted on the 
operator's chest detects the operator's movement to inform 
the computer 13 of it. The computer 13 arithmetically 
processes the output of the angle sensor 21 and the output of 
the angle sensor 107 provided in the display device 2, and 
if it determines that the output of the angle sensor 21 is 
equivalent to the output of the angle sensor 107, treats this 
as the movement of the entire body and allows the display 
on the virtual screen 6 to remain unchanged. That is, the 
relative locational relationship between the operator 1 and 
the virtual screen 6 is maintained constant, and the virtual 
screen 6 follows the movement of the operator's body. In 
this maimer, the virtual screen 6 is always displayed around 
the operator 1, and the virtual subscreen 5 can be viewed as 
required simply by looking somewhat upward, whatever 
posture the operator is assuming. 

Furthermore, pressing the push switch 14 while viewing 
a virtual subscreen 5, as described in Embodiment 1, enables 
that subscreen 5 to be specified on the basis of the direction 
of the operator's line of sight detected by a photosensor 109 
and the output of the angle sensor 107. Then, by moving the 
specified virtual subscreen 5 based on the output of both the 
photosensor 109 and the angle sensor 107 and subsequently 
pressing the push switch 14 again, the specified virtual 
subscreen 5 can be fixed to the current observation point. In 
this manner, an arbitrary virtual subscreen 5 can be moved 
to a desired position on the virtual screen 6. 

Next, it is assumed that the operator 1 is actually repairing 
the apparatus 22. Instruction manuals and specifications 
required for repairs are registered in the computer 13 in 
advance. A plurality of virtual subscreens 5 are displayed on 
the virtual screen 6, that is, stored in an image storage device 
133. A desired instruction manual can be viewed by moving 
the operator's head to display the corresponding virtual 
subscreen 5 within the display frame 51. Furthermore, by 
pressing the push switch 14 while viewing a desired virtual 
subscreen 5, moving the operator's head and Une of sight to 
a position in which the subscreen can be seen clearly, and 
then pressing the push switch 14 again, the virtual subscreen 
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can be fixed to the virtual screen 6. In addition, even if the 
operator moves to repair another part of the same apparatus, 
the virtual screen 6 remains in the same position around the 
operator 1, so required instruction manuals can be refer- 
enced whatever working posture the operator is assuming. 

As an alternative to the trigger means described in 
Embodiments 1 and 2, the output of photosensor 109 is 
obtained, for example, when the operator gives two con- 
secutive quick winks. This enables both hands to be used for 
repairs even in Embodiment 2. It is clear that voices or 
breathing can also be used as a trigger means. 

Embodiment 3 

The display device 2 in the embodiment shown in FIG. 10 
has a spectacles-like frame 101 including the liquid crystal 
panel 102, the enlarging lens 103, the back light 104, and a 
half mirror 135. The operator 1 can view an enlarged display 
image on the liquid crystal panel 102 and an external image 
(the keyboard) through optical paths 136 and 138, respec- 
tively. These images can be synthesized by the half mirror 
135 and viewed by the operator 1. The operator 1 thus feels 
as if the liquid crystal display image were present outside the 
frame (near the keyboard), resulting in improved workabil- 
ity. 

With this configuration, however, a few problems may 
occur. First, the liquid crystal display image and the external 
image simply overlap each other, so the display image 
cannot be distinguished clearly. Second, since iie distance 
between the operator and the virtual image cannot be 
matched easily with the distance between the operator and 
the keyboard, focusing for eyes must be carried out each 
time the field of view is switched between the liquid crystal 
display image and the keyboard contained in the synthesized 
image, resulting in a marked fatigue of the operator. 

These problems can be solved by employing part of an 
outer sheath case for the data processing apparatus and the 
keyboard as a screen. That is, the rear face 141 of a cover for 
the keyboard 140 of the computer 3 is used as a screen. This 
screen consists of an opaque material with a single color 
such as white, black, or blue, and is effective in improving 
the visibility of a liquid crystal display image that is allowed 
to overlap the screen image. The color and reflectivity of the 
screen are depends on the degree of this effect. 

As in the example shown in FIG. 12, the operator 1 can 
longitudinaly move relative to the operator 1, the position in 
which the virtual subscreen 5 for a liquid crystal display 
image is formed by moving the liquid crystal panel 102 in 
the direction 139 (along the optical path 136) in FIG. 10. 
Then, matching this position with the screen 141 enables the 
above problems to be solved. 

Since the external image that overlaps the liquid crystal 
display image is the monochrome screen, it does not sub- 
stantially disturb the visibility of the display image. That is, 
the display image has a high visibility, leading to improved 
workabiUty. In addition, the virtual subscreen 5 for a liquid 
crystal display image is formed on the screen 141 close to 
the keyboard 140, so focusing for eyes need not be executed 
when the field of view is switched between the display 
image (the virtual subscreen 5) and the keyboard 140, 
resulting in a reduced fatigue of the operator. 

Consequently, the above embodiment enables the imple- 
mentation of portable computers and word processors that 
have high portability and visibility and that do not affect the 
operator's health. 

The characteristics of the screen is not limited as long as 
they meet the objectives of this invention. For example, 
attaching a polarizing plate to the screen can reduce the 



7,935 

16 

brightness of the external image with the brightness of the 
liquid crystal display image maintained, thereby enabling 
high contrast display to be realized. 
Embodiment 4 

^ The display device in FIGS. 13 and 14 is of a single eye 
type. In these figures, the cable for connecting the drive 
circuit to the main body is omitted for simplification. 

An image displayed on the liquid crystal panel 102 is 
enlarged by the enlarging mirror 145, and the operator's 
right eye sees a large display image in front of the operator, 
while his or her left eye sees the exterior through the 
transparent spectacles-like frame 101. The problem in single 
eye-type display devices (the visibility of the display image 
is bad because the external image overlaps the display 
image, and the difference betweeri focusing for the right and 
left eyes causes a significant fatigue of the operator) can be 
solved by adjusting the distance between the liquid crystal 
panel 102 and the enlarging mirror 145 so as to form on the 
screen a virtual subscreen 5 that is a display image so that 
the operator's left eye can see the screen 141, That is, since 
the virtual screen on the liquid crystal display screen is 
formed on the monochrome and uniform screen 141, the 
visibility of the liquid crystal display screen is improved, 
and an equal focusing for the right and left eyes serves to 

^ reduce the operator's fatigue to almost the same level as in 
ordinary displays. 

Consequently, the above embodiment can provide por- 
table computers and word processors that have a good 

2P portability and visibility and that do not affect the operator's 
health. 
Embodiment 5 

In the embodiment in FIG. 15, a signal from the input 
device 150 is input to the computer (an arithmetic unit) 3 via 

35 the connection cord 151, and image display information 
output from the computer 3 is transmitted to the head- 
mounted display device 2 via the connection cord 106. The 
head-mounted display device 2 enables the simultaneous 
display of that input information such as the conditions of 

40 the input device 150 such as a keyboard or a data glove 
which is output by the computer 3 and the resuhs of 
arithmetic operations firom the computer 3, and indicates 
what keys can be currently used for input and what key the 
operator is touching. 

45 The display screen of the head-moimted display device 2 
is long, and is composed of an arithmetic operation result 
display screen 152 for displaying the results of arithmetic 
operations within the virtual subscreen 5 and a virtual 
keyboard display screen 153 for displaying the conditions of 

50 the input device. By using as the input device 150, for 
example, a data glove having a means for measuring the 
transmittance of hght through optical fibers to detect the 
bending of the operator's fingers and using a magnetic 
sensor to delect the relative position of the operator's hands, 

55 or a keyboard with a sensor having a position detection 
means such as a resistant fihn-like tablet, the input position 
can be detected, and the results of arithmetic operations can 
be viewed while a key assignment means 156 and an image 
synthesizing means 159 in the computer 3 are used to check 

60 on the virtual keyboard display screen 7 what key is being 
pressed. 

Next, a method for outputting the results of arithmetic 
operations on the aritmetic operation result display screen 
152 and displaying the conditions of the input device 150 on 
65 the virtual keyboard display screen 153 is described with 
reference to the block diagram in FIG. 17. The input device 
150 detects positional information 154 for the positions of 
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the operator's hands and fingers and on/o£f information 155 preceding signal (Xq, Yq) (S28). The assignment means 

indicating whether or not any finger tip is pressing a key to determines whether or not the difference is, for example, 2 

transmit this information to the computer 3. The computer 3 cm or longer (S29), and if so, treats this input as the result 

uses the key assignment means 156 to assign keys over the of cUcking on (Xq, Yq) (S30). If the difference is shorter than 

input range of the input device 150 for each application in 5 2 cm, the first signal is moved by the corresponding amount 

advance in order to determine what keys correspond to input movement. The above processing between step S26 and 

positional information 154, and determines from the on/off ^tep S31 is performed in 0.01 second or shorter, 

information 155 whether or not the key is activated in order . . . -.ri • 1 . • • t-,^ 

to transmit the results to an arithmetic operation section 158 ^ ^° ^° ^^fPj^y ^^^^^^ ^ m FIG. 15, 

and an image synthesizing means 159 as input information connecUon cord 106 is connected to the 

157. The arithmetic operation section 158 performs arith- ^pectacle-like frame 101 for mounUng the display device 2 

metic operations based on the input information 157 to ^^l^''^' ^ ^^^.^ '''r^^\!^.''^u^^^^^^' 

transmit the results 160 to the image synthesizing means \ I'^'^ ^t light 104 disposed 

159. The image synthesizing means 159 continuously out- ^^^^^^ ^^'^^ ^ecUcle-like frame 101 Aii image 

puts the conditions of the input device 150 such that which ^'^""^ \^P^^ ^^^^^ '"'l^'^'l^^J,'^ ^^^^ ^"^^^T^ 

key the operator's hand is touching based on the input on the liquid crystal panel 102. m d^^^^^^ 

information and the on/off state of the key, and outputs to the "^^^f ^ illummated by the back hght 104 disposed on the 

head-mounted display device 2 a signal as an image signal ^f^^ hquid crystal panel 102, and reaches the 

161 which synthesizes the results of arithmetic operation ^^''f' ^ iff'""^ ^^^f ^""^^'^'^^ 

160 from the arithmetic operation section 158 into one ^^^/^^^j^^^" ^^^^^^^ spectacle-hke 

image each time such an operation is carried out. TTie input f ^^^J^.' ^^Pf ^^^^^ ^y^^ °f °P^^^^f ' 

device 150 simply transmits the positional information 154 ^^^^^^ /^P^^*' f^^^^'^'l !k "P?'""' between the 

and the corresponding on/off information 155 to the com- 1^4^/^ ^^^tal panel 102 and the reflection ^urror 135 to 

puter 3, thereby enabling the key assignment means 156 to P«^J^^\ ^ ''f^^''' J^^'^""' ^^^^1^^. 

assign various keys without depending on the number of image to reach bo h eyes of the user via the reflecUon mirror 

keys on the input device 150 and also aUowing the virtual ^ and the enlarging lens located m front of the respective eyes, 

keyboard display section 153 to display various interfaces. The liquid crystal panel 102 is a long and small full color 

The flow of signals in the control system in FIG. 17 is or monochrome LCD with a screen size of 1 to 2 inches, and 

described with reference to FIG. 18. The input device 150 ^^^^^^ subscreen 5 about 14 inches in size to be 

detects the input position and the corresponding on/off 3. seen by the user via the edarging lens 103 at a distance 5(W 

information (SIO). The key assignment means 156 deter- to 1,000 mm apart from the user s eyes, as shown m FIG. 20. 

mines to what key the input position corresponds, and above embodiment, although the computer and the 

transmits the contents of the corresponding key to the ^^^^play device are separated from each other, the computer 

arithmetic operation section 158 (Sll). The key assignment may be mcorporatcd in the head-mounted display device, 

means 156 then determines whether or not the key is being 35 in this case, signals from the input device are transmitted 

pressed, and if so, informs the arithmetic operation section directly to the head-mounted display device and the com- 

158 of this state (S12). The image synthesizing means 159 P^*®^- computer may be implemented with an anth- 

then generates an image on the virtual keyboard display metic operation device designed for a display device, 

screen 153 based on the input position and the on/off The interface in the above embodiment for which the key 

information (S13). The arithmetic operation section 158 ^ assignment means in the arithmetic operation device 

performs arithmetic operations based on transmitted infor- executes assignment is not hmited to a keyboard but may be 

mation to generate an arithmetic operation result display a mouse, a joystick, or a pen. 

screen 152 (S14). The image synthesizing means 159 syn- Furthermore, the input device is not limited to a tablet or 

thesizes as a single screen the virtual keyboard display a data glove as in the above embodiment, various input 

screen 153 and the arithmetic operation result display screen 45 devices that can detect positional information and on/off 

152 generated by the arithmetic operation section 11 to information may be connected to the computer, 

generate a virtual subscreen (S15). As an alternative to the embodiment shown in FIG. 16, the 

The input device 150 includes resistant film-type devices, image synthesizing means in the computer (the arithmetic 

a coUection of smaU switching elements, and various other unit) can be omitted by separately driving and displaying the 

devices. Next, the process executed by the assignment 50 arithmetic operation result display screen and the virtual 

means 156 when a resistant film-type input device is used for keyboard display screen to thereby implement the display 

key and pointing input is described in detafl. section of the head-mounted display device. 

The assignment means 156 first determines the input Embodiment 6 

method of the input device 150(520). If the input device 150 The embodiment in FIGS. 21 to 29 shows the data 

uses a key input method, the assignment means 156 divides 55 processing apparatus according to this invention appUed to 

the input plane into key areas (S21). The assignment means a personal computer. FIG. 21 is a perspective view wherein 

subsequently determines whether or not the input method the display device 2 in the above embodiment is used as the 

has been switched (S22), and if not, receives the input signal display means. A signal of image display information output 

(X, Y) (S23). It then examines to which key area the input from the computer 3 is transmitted to the display device 2 

signal (X, Y) corresponds (S24), and outputs the code for the eo via the connection cable 106. To use the display means 

corresponding key (S25). separately from the method described below, the cable 106 

If the assignment means determines in S20 that the input is connected to the computer 3 via a connector 23 so as to 

device uses a pointing method, it then determines whether or be removed easily therefrom. 

not the input method has been switched (S26). If not, it The display device 2 in FIG. 21 essentiaUy has the same 

receives the input signal and stores it in a buffer (Xq, Yq) 65 constitution as the device shown in FIG. 20 except the Uquid 

(S27). It then receives the next input signal (X^i, Y^) to panel 102 can be removed easily from the frame 101. The 

determine the difference between the this signal and the reflection mirror 170 can be moved vertically and longitu- 
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dinally and also have its angle adjusted so that the operator 
can get a clear view. The frame 101 serves to intercept 
extraneous light so that the image displayed on the liquid 
crystal panel 102 can be seen clearly. The exterior, however, 
must be seen through the frame 101, so the frame 101 
preferably has a light transmittance of about 70 to 80%. 

In addition, in the embodiment in FIG. 21, power for 
driving the display device 2 and image information output 
from the main body are transmitted to the display device 2 
via a connector recess 60 provided in the computer 3 main 
body, a connector convex 23 provided on the display device 
2, and the connection cable 106 for connecting the computer 
3 main body to the display device 2. 

FIG. 22 shows a screen used as the display device, and 
FIG. 23 shows a cross section of the screen. 

The liquid crystal panel 102, removed from the display 
device 2, is plugged in a socket 24. An image displayed on 
the liquid crystal panel 102 is illuminated by light from the 
back light 25, enlarged by a lens 206, and projected on a 
screen 27. As a result, the operator can view the image 28. 
The vertical position and size of the image 28 can be 
adjusted by moving the liquid crystal panel 102 vertically or 
longitudinally in the socket 24. The screen is not necessarily 
a white panel, but may be a flat surface without patterns such 
as a wall. 

FIGS. 24 and 25 show the cor relationship between the 
vertical position of the liquid crystal panel and the vertical 
position of the image 28. These figures show that moving the 
liquid crystal panel 102 vertically enables the vertical posi- 
tion of the image 28 to be adjusted. 

FIGS. 26 and 27 show the correlationship between the 
longitudinal position of the liquid crystal panel and the size 
of the image 28. These figures show that moving the liquid 
crystal panel 102 longitudinally enables the size of the 
image 28 to be adjiisted. When the distance between the 
Uquid crystal panel 102 and the enlarging lens 26 and the 
distance between the enlarging lens 26 and the screen 27 are 
referred to as Sj and S^, respectively, expression (1) is given, 
(f) is a focal length. In addition, when the size of the image 
is referred to as p, expression (2) is given. 

l/^j-l/^j+l/Zj [EKpression 1] 

po^j/Sj [Expression 2] 

In expression (1), Sj increases with decreasing S^, while 
S^ decreases with increasing Sj. Consequently, the size of 
the image p increase with decreasing S^ while it decreases 
with increasing S^. 

FIG. 28 shows a perspective view of the structure of the 
socket 24, and FIG. 29 shows an exploded perspective view 
of the socket 24. 

The liquid crystal panel 102 has its vertical and longitu- 
dinal directions adjusted by a pin 54 and an eccentric pin 55, 
respectively. The liquid crystal panel 102 is inserted in the 
direction A in FIG. 29. The socket 41 fixes the liquid crystal 
panel 102. The pin 54 is inserted into a tapped hole 51 via 
a hole 42 to fix the socket 41 to a socket 44, Thus, the 
rotational movement of the pin 54 causes the socket 41 to 
move vertically and thus the liquid crystal panel 102 to move 
vertically. Pieces 46, 47 of the socket 44 engage the socket 
41, and the pieces and the socket 44 are fixed longitudinally 
together. Rotating the eccentric pin 55 in a long hole 50 
enables longitudinal movement. Pins 52, 53 are inserted into 
long holes 48, 49 to prevent lateral deflection. Windows 43, 
45 allow light from a Ught source to pass through. 
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Although the above embodiments have shown an example 
of a liquid crystal panel of a detachable structure, the liquid 
crystal panel in the display device 2 and the Uquid crystal 
panel in the computer main body can not necessarfly be 
5 removed from the respective devices, and the liquid crystal 
panels may be fixed and a display switch is provided to 
switch the display means for the computer. 

In addition, aUhough the above embodiment has referred 
to the xise of a personal computer, this invention is not 
limited to the above embodiments and is applicable to data 
processing apparatuses such as word processors. The single 
eye-type head-mounted display device has been described, 
but this invention is applicable to double eye- type head- 
mounted display devices. Furthermore, although the above 
embodiments have shown the use of a spectacle-type bead- 
le mounted image display device, this invention is applicable 
to other types of head-mounted image display devices such 
as goggle and helmet types. 

If the data processing apparatus of the above configura- 
tion is used together with a head-mounted image display 
20 device as a display means, only the operator can directly 
view the contents of display, so the operator can process 
secret information without caring about people aroimd the 
operator. The portability of this data processing apparatus is 
also good because the operator can use the apparatus in a 
25 natural position in a limited space as in an airplane or a train. 
In addition, the field of view can be moved easily between 
the virtual image viewed through the display device and the 
exterior including the keyboard, leading to the improved 
operability of the data processing apparatus. 
30 If a plurality of people desire to simultaneously see image 
display information output from the data processing 
apparatus, the liquid crystal panel in the apparatus main 
body may be used to display image display information on 
the screen or the like. Such switching of the display means 
35 can be carried out easily. 
Embodiment 7 

FIG. 30 shows a conventional computer in which its 
display section can be removed from its main body. A 
computer main body 3a is engaged with a display 63 via a 

40 connector recess 61 and a connector convex 62fl. 

FIG. 31 shows an embodiment in which the display 
device 2 in the above embodiment is mounted on the 
computer 3 in FIG. 30. In this embodiment, the computer 
main body. 3a is connected to the display device 2 via the 

45 connector recess 61, the connector convex 62, and the 
connector cable 106. 

. If both the connector convex 62a of the display 63 in FIG. 
30 and the connector convex 62 of the cable 106 in FIG. 31 
conform to the same standard, both the conventional display 

50 63 and the display device 2 according to this invention can 
be connected to the same main body. That is, a data 
processing apparatus is realized wherein the conventional 
display 63 and the display device 2 according to this 
embodiment can be exchanged easily. 

55 Although the above embodiments use a computer as the 
data processing apparatus, this invention is not limited to 
these embodiments, and is applicable to other data process- 
ing apparatuses such as word processors. The single eye- 
type head-mounted display device has been described, but 

60 this invention is applicable to double eye-type head- 
mounted display devices. Furthermore, although the above 
embodiments have shown the use of a spectacles- type 
head-mounted image display device, this invention is appU- 
• cable to other types of head-mounted image display devices 

65 such as goggle and helmet types. 

As is apparent from the above explanation, this embodi- 
ment is advantageous in that the display device can be 
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removed easily from the data processing apparatus. In with only the information to be monitored displayed on a 

addition, the use of connectors conforming to a common hcad-moxinted display device, 

specification enables exchanges between a conventional Embodiment 9 

display and the display device according to this embodi- In the display device in FIG. 35, a spectacles-like frame 
ment. Furthermore, the back light in the display device of 5 1^1 has mounted therein a liquid crystal panel display 
this embodiment consumes less power than displays used in section 172 for the left eye and a liquid crystal panel display 
conventional portable data processing apparatuses. As a section 173 for the right eye. These liquid crystal panel 
result, the life of the battery can be extended, resulting in the ^^Pj^y ^ecUons 172, 173 form a virtual image on the retina 
significantly improved portability of the data processing ^^^^ ^^^^ ^ ^^^f ^/^^ ^^^^^ .1^® 
apparatus according to this embodiment. lo P^^P^^^^^ to the strength of each image by irradiating the 
P K rt' t R ^^^^ liquid crystal panel 102 from behind with 
bmDocUmentS extraneous light or light from the back light 104. A drive 
In the image display system m FIG. 32, output results ^^^^-^ controlling these hquid crystal panels 102 may be 
from the computer 3 are displayed on the display 63and the configured integrally with or separately from the spectacles- 
display device 2 mounted on the operator s head. The user f,^^ ^o as to form images on the liquid crystal panel. 
1 can execute processing while slightly movmg his or her 15 A microphone 174 is joined with the frame with a flexible 
glance to view the image displayed on the display 63 and the joint 175 that is fixed to the position of the user's ears and 
virtual subscreen 5 created by the display device 2 mounted which can be bent toward the user's mouth. A tracking 
on his or her head. The virtual subscreen 5 is generated by sensor 176 that acts as a position detection means is installed 
allowing an image displayed on the liquid crystal panel 102 in the center of a frame 177 disposed so as to link the user's 
to reach the user's eyes via an enlarging reflection mirror 20 right and left ears together. The tracking sensor 176 detects 
section 64. the direction of the operator's Une of sight using the moment 
In the control system of the display device in FIG. 33, an of inertia effected when the user's head is turned. The 
output image from the arithmetic unit 70 is assigned by an display device in this embodiment includes a mouse 180 and 
output information assignment means 72 to the respective ^ keyboard 181. 

image output devices for output, and then written to a 25 ^^OS. 38 and 39 show an embodiment in which a still 

graphic RAM 73 in an image output control section 71. The Pi^^^^r^ obtained using the display device shown in FIG. 35 

information written to the graphic RAM 73 is managed on is edited These figures describe a display method in which 

the basis of addresses, information output from the arith- ^.^^ ^P^^y ^'^^^ <hsphy^d laterally on the 

metic unit 70 is output to the display device 2 and the display i n * ■ .-n • ^ . 

i.if ' c uj-ij^ The first display area usually contains still pictures that 

t 'l^u "^"J"", T ""^^ ? shown in front of the opera^or. The second display area 

both the display device 2 and the display ^ ^^^^ ^ ^^^^^ ^^^^^.^^ 

sets may be displayed on the display device 2 and the d^play ^^^^^^ t^e user's movement detected by the tracking 

63, respectively, and the information can also be transferred sg^sor 176 when he or she turns to the right. The second 

to an arbitrary image display device. display area displays attributes of the image data shown in 

The flowchart in FIG, 34 shows a process of outputting to 35 the first display area. In this case, when the user turns to the 

a plurality of image display devices image information right, the tracking sensor 176 detects this movement to 

output from the arithmetic unit. As if Image display devices create on the two liquid crystal panels 102 an image such as 

of apparently large capacities were connected to the arith- shown in FIG. 39. This embodiment relates to the synthesis 

metic unit, output information from the arithmetic unit is of still pictures, in which the segments of the image 

read into these display devices (S51). It is determined 40 (geometric models such as a tree, a house, or a car) are 

whether the read information comprises an image or controls detected on the second display area and have their sizes 

display areas (S52). If the information is to control the adjusted. In this case, for example, a "car" is input from the 

display area, a memory address area that must be set in the keyboard 181 to search its design file for an appropriate 

graphic RAM 73 to display an image on each display device material. Searches are similarly carried out for the house and 

is determined on the basis of this information (S53). If the 45 tree, and the size of the image is also adjusted on the second 

information comprises an image, parameters such as a display area. The processed images materials are transferred 

display area (addresses) are determined on the basis of the to the first display area in which they are synthesized against 

memory address area (S54). It is then determined whether or a background such as a mountain. 

not the image information should be contained in the display Next, the above editing process is described with refer- 

area determined in step S54 (S55), that part of the informa- so ence to the flowchart in FIG. 40. The output of the tracking 

tion which is not associated with the display area is ignored sensor 176 is first input, and the direction of the operator's 

during processing (S56) and the rest of the information line of sight is detected on the basis of this input signal 

which should be contained in the display area is address- (S20). If the fine of sight is determined to be to the left, 

converted (S57) and written to the graphic RAM (S58). Each background images are input to the first display area (S21) 

of the image display device displays the information in the 55 to select among them (S22). It is then determined on the 

assigned address area as required, basis of the output of the tracking sensor 176 whether or not 

The user views the virtual subscreen 5 in part of his or her the operator 1 has turned to the right (S23), and if not, the 
field of view and the other image display devices in the rest background image is then processed (S24). Until the pro- 
of his or her field of view, and can see the plurality of image cessing is finished (S25), the process of selecting and 
display devices simply by slightly moving his or her line of 60 processing a background images is repeated (S22 to S24). 
sight. When the processing has been finished (S25), it is then 

Although, in the above embodiment, two image display determined whether or not to add data to the image (S26). If 

devices are connected to the portable computer, the numbers it is determined that data should be added, attributes data is 

of arithmetic units and image display devices are not limited, added to the first display area (S27). 

and in a system that processes a large amount of information 65 Voice input is then used to edit the details of the layout 

using a plurality of arithmetic units, the results of processing (S28). This process is described later. When the edition is 

can be displayed on the plurality of image display devices finished (S29), the series of processing is completed. 



06/13/2002, EAST Version: 1.03.0002 



5,977,935 

23 24 

On the other hand, if the direction of the operator's line personal computers or personal computers. This arithmetic 

of sight is determined to be to the right when the direction unit 81, however, does not include a conventional open- type 

of the line of sight is detected (S20), attributes data is input display but comprises an arithmetic unit main body 82 

to the second display area (S30). A reference coordinate consisting of an arithmetic section, a memory section, and a 

position is determined (S31), and attributes data is searched 5 keypad; and a cover 83 for protecting the upper surface of 

for (S32). When the search is finished (S33), the attributes keypad during storage or transportation. As in conven- 

data IS processed (S34). When the processing is finished, it tional data processing apparatuses, the cover 83 is connected 

IS detenDined whe^ or not edition will be subsequenfiy arithmetic unit main body 82 via a hinge-like con- 

earned out (S36). If so, segments to be edited are selected „^^*:„„ 00 ui j 1 ui * ■ j t-u 

(S37). It is then determined whether or not the operator 1 is ^ ""^^ ? l l^ f J^" 

looking to the left, and if so, the reference coordinates are f f f f ^ \ ""^^ 

calculated (S39) and the process transfers to the above step f ^j^^^^^. ^""^l P^^^ f 1^ f^rms a housing 

S27. Otherwise, that is, if the operator is determined to be section 86 for housing the display device 2. The display 

still looking to the right, the process returns to step S34 to ^^^^^^ ^ composes a spectacles-like frame 101 in which the 

repeat the process of processing attributes data and selecting '^P^i^^l ^yst^m and drive circuit are housed, and is 

segments to be edited. 15 mounted on the user's head so as to cover his or her face. A 

An audio editing process is described with reference to t)ow 88 is formed at both ends of the frame 101 so as to 

the flowchart in FIG. 41. Audio input does not simply consist extend to the user's ears with hinges 89, and can be folded 

of understanding words implied by commands but employs and housed. A communication cable 106 from the drive 

an audio input means called "WordSpotting" for unspecified circuit protrudes from one end of the bow 88 and connects 

speakers in which clauses can be input. 20 to the arithmetic unit main body 82 through one end of the 

In this audio input means, sentences are understood by housing section 86. 

ncuro-computer processing in which nouns such as a tree, In operation, when the head-mounted image display 

the right side, movement, the left, and a height are registered device 2 is fixed to the user's head via the frame 101, and 

in advance to understand the entire sentence based on the the arithmetic unit 81 is driven, then the drive circuit and the 

occurrence probabiUty of each word. 25 optical system in the display device 2 enlarge an image 

In this case, a virtual image is shown within the first information output from the arithmetic unit 81 and display 

display area via a wide angle lens, so the image is enlarged, it as a virtual image at a definite distance from the operator's 

and it is possible to allow the user to feel as if he or she were face. In storage or transportation, the bow 88 on either side 

in that image. The distance over which the image moves is of the display device 2 is folded and the display device is 

set on the computer to the size of the first display area, and 30 housed in the housing section 86. The communication cable 

the distance between the point of view and an object is 106 is also folded and housed in the housing section 86, and 

calculated, in advance. During edition, a desired image can the cover 83 is closed. This apparatus then constitutes an 

be input in the form of a voice to edit even minor positional integral structure like conventional portable data processing 

relations while viewing the image in the first display area, apparatuses. Consequently, it is possible to improve the 

without changing the point of view. 35 portability and operability of the data processing apparatus 

The point of view can also be changed in the second while maintaining conventional advantages such as the 

display area as if this change were effected by camera work. ability to prepare confidential documents and to avoid 

FIG. 42 shows a simulation of motion picture data using locational limitations such as the need to fully open the 

the display device shown in FIG. 35. A simulated image (a) display. 

is displayed in the first display area, while its attributes are 40 The operation of the above embodiment is explained with 

displayed in the second display area. The second display reference to the block diagram in FIG. 44. An electric 

area shows spatial area data that varies with the lapse of current supplied from an external power supply AC input is 

time, for example, temporally changing and graphed indi- allowed to branch into two flows at a power circuit 96 in the 

cations of the (b) locus of a flying airplane, (c) its speed, and arithmetic unit 81, one of which is fed to the arithmetic 

the (d) hierarchical segmentation of the image segments. 45 operation section 97. The other flow and the output of the 

In this embodiment, the locus of the airplane can be arithmetic operation section 97 are supplied to the drive 

obtained by plotting the temporally changing location of the circuit 105 in the display device 2 via the connection cable 

airplane on the second display area. A speed can be set for 106, and then converted into a liquid crystal driving signal 

each plotted point based on the time vs. speed graph. If, for to drive the liquid crystal panel 102. In this block diagram, 

example, the speed is to be varied, the mouse is clicked on 50 components that are not required for the explanation are 

each step point to move it upward or downward. The omitted and are not described. In addition, the size and 

segments that appear in the image data according to a certain weight of the display device 2 can be further reduced by 

time base can be cut or pasted to the image based on the dividing the drive circuit 105, which is housed in the display 

graph in which the image segments are hierarchically device 2 in this embodiment, into two parts, one of which is 

expressed. 55 incorporated in the arithmetic unit 81 in such a way that this 

Such processing enabling the attributes of the image data configuration will not result in the occurrence of noise, 

to be temporally varied is very effective in editing seg- Embodiment 11 

mented data during postproduction. The efficiency of edition The data processing apparatus in FIG. 45 has the same 

is further improved by applying the depth of each image basic configuration as in the preceding embodiment. In this 

material and a paraUax that corresponds to the difference 60 embodiment, a housing section 90 for housing the display 

between the right and left eyes in order to achieve three- device 2 is formed inside the arithmetic unit main body 82, 

dimensional display in which the operator feels as if he or and the connection cable 106 is connected to both the power 

she were viewing actual scenes as well as hierarchical supply in the arithmetic unit 81 and the arithmetic operation 

display that expresses the depth well. section. In this constitution, although the volume of the 

Embodiment 10 65 arithmetic unit main body 82 main body somewhat 

In the data processing apparat\js in FIG. 43, an arithmetic increases, the need of a large cover is eliminated to reduce 

unit 81 is a portable type such as conventional notebook the size and weight of the apparatus and thereby improve 
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portability, and the need of an open-type cover is eliminated 
to reduce the locational limitations, thereby further improv- 
ing usability. 
Embodiment 12 

The data processing apparatus in FIGS. 46 and 47 also has 
the same basic configuration as in the two preceding 
embodiiments. Like components carry like reference 
numerals, and their description is thus omitted. 

In this embodiment, the arithmetic unit 81 has a charger 
therein, and the display device 2 has a chargeable battery 
therein. In addition, the arithmetic operation section 97 and 
the drive circuit 105 communicate with each other by radio. 
The use, storage, or transportation of this apparatus thus 
requires a smaller space. The arithmetic unit 81 has a charger 
98 to which electric currents are supplied from a power 
supply 96 and a transmission circuit 99 to which signals are 
delivered from the arithmetic operation section 97. A signal 
supplied to the transmission circuit 99 is modulated in the 
circuit 99, then transmitted and received via antennas 201, 
202, and demodulated in a receive circuit 203. The signal is 
finally input to the drive circuit 105, The display device 2 
has a chargeable battery 204 therein for delivering drive 
currents to the drive circuit 105. 

An electrode contact 205 cormected to the charger 98 is 
exposed on the end face of the housing section 90. If, for 
example, the head-mounted image display device 2 is to be 
housed in the housing section 90 by inserting the antenna 
202 side into the section 90, an electrode contact 206 is 
provided at the antenna 202 -side end of the frame 101 that 
engages the electrode contact 205 when the display device 
is housed in the main body, and connected to the battery 204. 
In this configuration, when the head-mounted image display 
device 2 is housed in the main body, the electrode contact 
205 of the charger 98 and the electrode contact 206 of the 
battery 204 are connected together. The battery 204 is thus 
charged while the display device 2 is housed in the main 
body, that is, while the device 2 is out of use. 

In this constitution, the arithmetic unit 81 and the display 
device 2 are completely separated from each other, thereby 
substantially simplifying the use of the data processing 
apparatus 81 as well as the mounting and removal of the 
display device 2 from the housing section 90. In addition, 
even when the display device 2 is housed in the same 
manner as in the embodiment in FIG. 43, similar effects can 
be produced by adding the above components to the appa- 
ratus. 

The display device in each of the above embodiments 
need not necessarily be of a single eye type in which an 
optical system is disposed over only one eye while the other 
eye views an image, but may be of double eye type in which 
an optical system is disposed over each eye. In this case, the 
optical axis of each optical system is preferably directed 
inward so that the glances of the two eyes intersect each 
other at an image visible distance (a distance at which a 
virtual image can be perceived), as in the natural way of 
looking at objects. In addition, the above components are not 
limited to the above contents, and various modifications may 
be made thereto without departing from the spirit of this 
invention. 



7,935 

26 

What is claimed is: 

1. Ahead-mounted image display device wherein a hquid 
crystal display panel and an enlarging optical system are 
housed in a frame and wherein images generated on said 

5 liquid crystal display panel are visible through said enlarg- 
ing optical system in the field of view in front of said frame, 
comprising: 

a first direction detection means for detecting a direction 
10 of the head of a user of said device; 

an observation point detection means for delecting said 
user's line of sight; 

an image generation means having functions for forming 
a virtual display screen displaying a plurality of virtual 
images in the field of view in front of said frame, 
scrolling said virtual display screen based on an output 
of said first direction detection means, selecting one 
virtual image from the pluraUty of virtual images on 
2Q said virtual display screen based on an output of said 
observation point direction means, and moving the 
selected virtual image based on outputs of said obser- 
vation point detection means and said first direction 
detection means; and 
^ a trigger input means for determining the selection of one 
virtual image and the finish of moving said selected 
virtual image, in said image generation means. 

2. A head-mounted image display device according to 
claim 1, wherein said image generation means further has a 
function for enlarging or reducing said selected virtual 
image determined by said trigger means. 

3. head-mounted image display device according to claim 
1, further comprising; 

35 a second direction detection means for detecting the 
direction of said user^s body, wherein scrolling said 
virtual display screen is forbidden based on an output 
of said second direction detection means. 

4. A head-mounted image display device according to 
40 claim 1, further comprising: 

a synthesizing optical means for synthesizing an image 
showing the exterior of said user and the virtual display 
screen visible through said enlarging optical system. 

5. A head-mounted image display device according to 
claim 4, further comprising: 

a real screen disposed in the direction in which enlarged 
virtual images are visible through said enlarging optical 
system, wherein said image showing the exterior of 
50 said user is the image of said real screen. 

6. A head-mounted image display device according to 
claim 1, further comprising: 

a connector for connecting the image display device to an 
image processing apparatus including a central pre- 
ss cessing unit and a keyboard. 

* )tt % )|C 4c 
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